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ABSTRACT: 


The objective of this study was to explore the use of multi-source remotely sensed data for individual tree species. To achieve this, a 
neutrosophic logic-based method was developed for tree species classification using the combined spectral, textural and structural 
information derived from WorldView-2 (WV-2) multispectral bands, WV-2 panchromatic band, and LiDAR (Light Detection And 
Ranging)-derived canopy height model (CHM), respectively. The developed method was tested on the data obtained over the Keele 
campus, York University, Toronto Canada and the KNN (K Nearest Neighbour) classification method. Twenty-one spectral, three 
textural and three structural features were used to classify five species (Norway maple, honey locust, Austrian pine, blue spruce, and 
white spruce). For this study, 522 trees were used for training and 223 for testing. The overall classification accuracy obtained by the 
proposed method was 0.82. It was significantly improved compared with the KNN (0.73), weighted KNN (0.76), and fuzzy KNN 
(0.75) methods. In addition, Dempster-Shafer (DS) theory was explored to perform information fusion at the decision level in 
comparison to that at the feature level. The accuracies obtained by the fusion at the decision level were generally lower than those at 
the feature level. Even though promising results based on the neutrosophic logic were obtained during this proof-concept stage, studies 
are underway to perform more tests with a large number of tree crowns and more species and exploit other classification methods, such 


as support vector machine. 


1. INTRODUCTION 


Trees are essential components of urban ecosystems and provide 
a wide range of environmental, ecological, social, cultural and 
economic benefits. Accurate tree species classification is 
therefore important to city planning, ecological management, and 
other urban studies (Iovan et al., 2008). The recent advance in 
remote sensing technologies makes a huge amount of data from 
different sensors, such as high spatial resolution imagery and 
high point density LiDAR (Light Detection And Ranging), 
readily available. The high spatial resolution data allow one to 
take the advantage of the spatial and structural features of 
individual tree crowns, in addition to the commonly used spectral 
features (Zhang and Hu 2012, Alonzo et al. 2014, Li et al., 2015, 
and Fang et al., 2018). Furthermore, considering individual tree 
crowns as the basic units provides a flexible platform to integrate 
information from different sources of data. It is relatively easier 
to register data at individual crown levels, rather than at the 
individual pixel level. Multi-source remotely sensed data also 
challenge researchers to develop effective methods to utilize fully 
all available information of individual tree crowns in species 
classification. 


Data obtained from different remote sensors are redundant, since 
they cover the same area. They are also complementary, because 
different sensors measure different physical properties of 
individual tree canopies. In addition, information contained in 
individual data sources is often imprecision and uncertain. Fusion 
of redundant and complementary data provides a complete 
description of a given canopy, while reducing imprecision and 
uncertainty. Even though studies have showed the potential to 
combine features from multi-source remotely sensed data in 
improving tree species classification (Fassnacht et al. 2016), 
further investigation is required for a full understanding of the 
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discriminant powers of individual data set and to seek an efficient 
and effective way to combine information from different sources. 
This was the focus of this study. 


Fusion of data from several sensors usually occurs at either the 
feature or the decision level (Fassnacht et al, 2016). In a feature- 
level fusion, the feature sets derived from multi-sensors are 
consolidated into a single feature set. Machine learning methods, 
such as support vector machine (SVM) and random forest (RF), 
are commonly used to classify of the species of interest based on 
the combined features. SVM and RF are generally shown to be 
more robust for classification with a large number of features, in 
comparison with traditional parametric methods, such as 
maximum likelihood classification (Maxwell et al., 2018). 
However, the high dimension in the feature space resulted from 
the feature fusion is likely to be a concern for the applications 
where the size of training samples is small (Maxwell et al., 2018). 
In addition, the features derived from different data courses are 
usually treated equally in SVM and RF, even though some of the 
data sources may be more reliable than others. 


In contrary, in a decision-level fusion, each data source is 
analysed separately, and the uncertainty and imprecision 
associated with each data source in the decision-making is 
measured and considered in the fusion process. Many 
mathematical concepts or formalisms can be used to measure 
uncertainty and imprecision and perform decision fusion, such as 
probability (Smets, 1993), fuzzy logic (Lucas and Araabi, 1999) 
possibilities (Dubois, 1986) and evidence theory (Dempster, 
1967 and Shafer, 1976). Among them, the Demspter-Shafer (DS) 
theory provides a mean to explicitly handle uncertain, imprecise, 
and ignorance, and it is commonly used in remote sensing 
applications. The DS theory, developed by Dempster (1967) and 
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Shafer (1976), is considered as a generalization of the Bayesian 
theory. With the DS theory, both inaccuracy and uncertainty in 
decision-making are represented through the definition of belief 
and plausibility functions that are calculated from a mass 
function. For a given evident, a mass function indicates the 
degree of belief in a proposition (also called as a focal element), 
given a piece of evidence and the belief can be associated with a 
simple proposition (a single class) or a compound proposition (a 
union of classes). The most crucial step in the application of the 
DS theory is the determination of the mass function. The 
commonly used methods in defining a mass function are based 
on a probabilistic distribution, mostly a Gaussian distribution 
(Walley, 1987), and the fuzzy logic (Waltz and Buede, 1986). 
Compared with probability-based approaches, a fuzzy approach 
has the advantage in handling inaccurate values and thus 
improves the mass distribution assigned to a compound 
disposition (Germain et al., 2002). Both probability-based and 
fuzzy approaches have been used in remote sensing image 
classification (Jouan, Allard, 2004, Mora et al, 2011, and Aval et 
al, 2019). Even though promising results are reported, 
classification accuracies could be further improved by explicitly 
addressing the uncertainties associated with training samples and 
individual data sets in information fusion. 


Most of the existing methods in information fusion for 
classification are based on supervised learning. As in any 
supervised learning method, the quality of training samples is 
critical. It is common that some training samples may be 
problematic. However, most of the methods treat each training 
sample equally. To improve classification accuracy, it is 
important to develop a strategy to evaluate the quality of each 
training sample and consider it in the supervised learning process. 
In this study, the quality of training samples was characterized 
based on the neutrosophic logic that was created by Smarandache 
(1999) and considered in the determination of the mass function. 
In neutrosophic logic, a logical variable is associated with three 
components, the degrees of truth, indeterminacy, and falsehood. 
It can be used to uncertainty and ambiguity in data and 
mathematical models. The neutrosophic logic has been employed 
mostly in the segmentation of remotely sensed imagery (Naveed 
et al., 2019). To the best of knowledge, it has not been used for 
tree species classification or broadly land cover classification. In 
this study, the discriminant power related to individual data 
source is also explicitly accounted for in the combination rule. 
The proposed strategies were tested using K Nearest Neighbor 
(KNN) classifier and data collected over the Keele campus of 
York University, Toronto, Ontario, Canada. 


2. STUDY AREA AND DATA USED 


The study area is located in the Keele campus of York University, 
Toronto, Canada (43.7735° N, 79.5019° W). Several remotely 
sensed data over the campus are available for this study. They 
include the Worldview-2 (WV-2) imagery obtained on July 21, 
2016, and airborne LiDAR data acquired in April 2015 with a 
point density of 10 points per square meters (Airborne Imaging, 
2015). The WV-2 imagery included one panchromatic band with 
a spatial resolution of 0.4 m by 0.4 m and eight multispectral 
bands with a spatial resolution of 1.6 m by 1.6 m. The WV-2 
imagery was converted to surface reflectance. The LiDAR data 
were collected using a Leica ALS 70 LiDAR instrument at a 
flying height of 1300 meters with a Pulse Rate Frequency (PRF) 
of 400 kHz. The horizontal accuracy of the collected LiDAR data 
was 30 cm and a vertical accuracy (on flat hard surfaces) of 10 
cm. A digital elevation model (DEM) and digital surface model 
(DSM) with the same spatial resolution as the WV-2 


panchromatic band were subsequently generated from the 
LiDAR data cloud. The canopy height model (CHM) was 
derived as the difference between the DSM and DEM, and then 
smoothed with a 3 by 3 Gaussian low-pass filter to eliminate 
noise. The ortho-rectification method in ENVI software was used 
to co-register the WV-2 imagery to the CHM via LiDAR 
intensity image and LiDAR-derived DSM. Tie points were 
manually selected, and they were at various elevations (on the 
ground or on buildings) and evenly distributed across the study 
area. The resulting co-registration accuracy was at 0.57 pixels 
based on 400 tie points. The false colour composite of the WV-2 
imagery and the LIDAR CHM are shown in Figure | and Figure 
2, respectively. 
79°3 1'0"W 
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Figure 1: The false colour composite of the WV-2 imagery of 

the study area with the near-infrared band printed as red, red as 
green and green as blue, respectively. 
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Figure 2: The CHM derived from the airborne LiDAR data over 
the study area. 


There are trees of various species along roads and in woodlots. 
The six common species were selected for this study and they are 
Norway maple (Acer platanoides), honey locust (Gleditsia 
triacanthos), Austrian pine (Casuarina equisetifolia), blue 
spruce (Picea pungens), and white spruce (Picea glauca). A 
sample database of 745 trees was randomly selected based on 
street tree inventory surveyed by the Campus Services and 
Business Operations of York in June 2015. The selected tree 
samples were located along streets, near buildings and other high 
pedestrian areas, thereby representing the typical distribution of 
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trees in an urban area. The sampled trees included 187 Norway 
maple, 180 honey locust, 159 Austrian pine, 120 blue spruce, and 
99 white spruce. 522 of the 745 tree samples (around 70%) were 
used for training and the rest (223 trees) for test. The crowns of 
these trees were delineated manually on the CHM by using 
information presented by both WV-2 imagery and LDAR data. 
An example of an individual tree crown appeared in these images 
is shown in Figure 3. 


Figure 3: An individual tree crown appeared on the false colour 
image of the WV-2 multispectral bands (left), WV-2 
panchromatic band (middle), and LDAR-derived CHM (right). 


3. METHOD 
3.1 Feature extraction 


Three categories of features were derived for species 
classification, including 21 spectral features from the WV-2 
multispectral bands, 3 textural features from the WV-2 
panchromatic band, and 3 structural features from the LIDAR 
CHM image. These features were calculated for each tree crown. 


The spectral features included the mean and standard deviation 
of the reflectance value in each of the eight spectral bands. In 
additional, five commonly used vegetation indices were 
generated to supplement the spectral information. As shown in 
Table 1, they were Normalized Difference Vegetation Index 
(NDVI), Enhanced Vegetation Index (EVI), Optimized Soil 
Adjusted Vegetation Index (OSAVI), Green Normalized 
Difference Vegetation Index (GNDVD, and Red Edge 
Normalized Difference Vegetation Index (RNDVJD. 


Name Equation 

NDVI (Puri — Prea)/(Pniri + Prea) 

EVI 1.6 (Pwiri — Prea)/(Pwiri + Prea + 0-16) 

OSAVI 2.5(Pniri — Prev)/(Pniri + ©Prea — 7-5 
* PBlue le 1) 

GNDVI (Pwie1 — Pcreen)/(Pniri + Pcreen) 

RNDVI (Preazage — PRed )/ Preazage ee Prea) 


Table 1: Vegetation indices used for species classification. p 
refers to the reflectance and the subscript corresponds to the 
WV-2 band (Jensen, 2007) 


The three texture features were derived from the WV-2 
panchromatic band. The spatial arrangement of leaves and 
branches within tree crowns may be different for different tree 
species and it is likely to form the textural variation in grey tones 
of the panchromatic image. For instance, Norway maple exhibits 
a smooth and fine texture, while locust tree appears relatively 
coarser. In this study, three GLCM (gray-level co-occurrence 
matrix) measures, contrast, energy, and homogeneity were used 
(Haralick et al., 1973, and Ulaby et al., 1986). 


Three structural features, the mean and standard deviation in 
height and area, were calculated to reflect the 3-D characteristics 
of tree species at the crown level. Even though age, surrounding 
condition and competition in urban environment all affect actual 
canopy height of trees, these derivatives predict structural 


arrangement of tree elements in a crown at both horizontal and 
longitudinal directions. 


3.2 Neutrosophic sets of training samples 


The neutrosophic logic was used, in this study, to explore the 
relationship of a given training sample to the five species classes. 
As described earlier, in the neutrosophic logic, the degrees of 
truth, indeterminacy, and falsehood are associated to a logical 
variable. The neutrosophic set of a training sample i was defined 
as (7; pl i Fi), where T;; was the degree of truth of train sample i 
to species class j, and J; and F; were the degree of indeterminacy 
and falsehood of this training sample, respectively. 


For a given category of features (spectral, textural, structural, or 
combined), denote the mean vector and variance and co-variance 
matrix of the species class j as mjand X; and the feature vector 
for a training sample i as X;. The Mahalanobis distance from this 
sample to the centre of the species class j was calculated 
(Equation (1)). Euclidean distance was also investigated, but the 
Mahalanobis distance performed better. 


d3; = (%; — 7)" 35 1(%, — 7i,) (1) 


Inspired by the transformed divergency used in feature selection 
(Jensen, 2007), the distance between a training sample i to the 
centre of a class j was converted to a new measure m;, using 
Equation (2). The new measure was between 0 and 1. The closer 
a sample was to a class, the larger this measure. 


mi; = e7[aij-mi (a})] (2) 
where min (d? )= the minimum distance of all training 


samples to the class j 


Assume that the training sample i belong to species class k. The 
neutrosophic set for this sample was calculated based on 
Equations (3). 


Tij = my 
i =m 
3 
={ Mig qzé#k GB) 
a 1 qz=k 


where mj,= the largest measure from this sample to the 
centres of the species classes excluding class k 
Mjq= the smallest measure from this sample to the 


centres of the species classes 


The membership measuring the likelihood of the training sample 
i belonging to the species class j was calculated using Equation 


(4). 


Tij Mix > Miz 
mi =4TyQ-1) mz S mi (4) 
0 Fp=1 


3.3. KNN classification based on the neutrosophic logic 


KNN classifies an unknown tree crown based on the properties 
of its K closest training samples. In this study, the KNN classifier 
based on the neutrosophic logic (referred as neutrosophic KNN, 
hereafter) was implemented. The membership of an unknown 
tree crown s associated to any species class j was calculated 
using Equation (5). 
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Lh, wimy 
Ms; = 5 
o rE Wi ( ) 
w;= the inverse of the Euclidean distance between the 
crown Ss and training sample i. 


where 


In this study, K was empirically determined as 10. In addition, 
the weighted KNN (Dubani, 1976), and the fuzzy KNN (Keller 
et al., 1985) were also implemented, for comparison. With the 
same definition as in Equation (5), for the weighted KNN, the 
membership of training sample i to the species class to which it 
belonged was | and 0 for other classes. For the fuzzy KNN, the 
membership of training sample i to the species class j was 
defined as the inverse of the distance calculated by Equation (1). 


3.4 KNN classification based on the DS theory 


In the DS theory, a frame of discernment 0 is defined to represent 
all the classes under consideration: © = {0;, 02, ...0y} and the 
power set 2® is then defined to include all the subset of © and the 
empty set @. The mass function within the range of [0, 1] is used 
to indicate the degree of belief in a hypothesis, given a piece of 
evidence. For each hypothesis in 2°, A; (denoted as A; & 2°), 
the mass function m(A;) satisfies the following conditions: 


m(A;) € [0,1]; m(@) = 0; Yajco2 M(Ai) = 1 (6) 
The aggregation of multiple belief functions from all of the 
available evidences is called Demspter’s orthogonal sum or 
Demspter’s rule of combination in the DS theory and it is 
calculated from evidences (1,2, ...,) in the following manner. 


N 
LByn..nBy=Alli=1 MiB) 


1-Yipin..nB y=ol es MiB) 


M1 2,..n(A) = (7) 
However, when the conflict among the evidences is larger, that 
is, the denominator in Equation (7) equals close to zero, the 
combination rule provides counter-intuition decision. To 
overcome this issue, different combination rules have been 
proposed. Among them is the Murphy’s averaging rule (Murphy, 
2000). In this study, we employed a weighted averaging rule and 
the weight was determined by using KNN classification. The 
memberships obtained from the KNN classifiers described in 
section 3.3 were used as mass functions. 


4. RESULTS AND DISCUSSION 


and textual features (i.e. information fusion at the feature level) 
achieved higher accuracies than those using individual features. 
The proposed neutrosophic KNN outperformed both weighted 
KNN and fuzzy KNN and obtained an overall accuracy of 0.82 
using the combined features. 


With the memberships of test samples belonging to each species 
class calculated based on individual features (spectral, structural 
and textural), the classification accuracies obtained by the 
information fusion at the decision level based on DS theory are 
shown in Table 3. From Table 3, it is clear that the spectral 
features had the highest the power in the discriminating the five 
species of interest and the lowest one was from the textural 
features. By combining these three groups of features using the 
proposed method, the classification accuracies were increased. 
The highest accuracy was achieved by using weighted KNN 
(0.79) and the lowest accuracy was by fuzzy KNN (0.64). 
Compared with the results obtained by the fusion at the feature 
level (Table 2), one can see that for both fuzzy KNN and 
neutrosophic KNN, the feature level fusion outperformed the 
decision level fusion. On the contrary, the decision level fusion 
based on weighted KNN performed better than the fusion at the 
feature level. The low discriminant power by the textual features 
may complicate the results. Investigations are being undertaking 
to improve the classification accuracies by adding more textual 
features and by using more advanced classification methods. 


KNN methods 
weighted fuzzy neutrosophic 
Spectral 0.70 0.61 0.67 
Structural 0.64 0.57 0.55 
Textural 0.51 0.38 0.37 
DS theory 0.79 0.64 0.77 
Table 3: Overall classification accuracies obtained using 
different KNN methods based on information fusion at the 


decision level. 


To further examine the classification results, the confusion 
matrices obtained by the neutrosophic KNN based on the fusion 
at the feature level and decision level are provided in Table 4 and 
Table 5, respectively. For both approaches, misclassification 
mostly occurred between blue spruce and white spruce. This 
might be due to the fact that these two species belong to the same 
genus and have similar properties. In addition, consistent good 
results were obtained for both Norway maple and honey locust. 
The major discrepancy between the two different fusion 
approaches lied on the classifications of Austrian pine. 


Reference 
The overall classification accuracies for the weighted KNN, MN | LH PA SB SW Producer’s 
fuzzy KNN and neutrosophic KNN based on spectral, structural, & accuracy 
and textual features individually and combined are shown in ‘S| MN 48 2 0 0 0 0.96 
Table 2. E| LS 7 47 1 5 2 0.75 
a PA 0 0 43 1 3 0.91 
KNN methods OD) SB 0 2 4 24 4 0.71 
weighted | fuzzy neutrosophic SW 0 3 0 6 21 0.70 
Spectral 0.70 0.61 0.67 User’s 0.87 | 0.87 | 0.89 | 0.67 | 0.70 | OA: 0.82 
Features | Structural | 0.64 0.57 0.55 accuracy 
Textural 0.51 0.38 0.37 Table 4: The confusion matrix for the neutrosophic KNN using 
Combined | 0.76 0.75 0.82 the combined features (feature level fusion), where MN, LS, 


Table 2: Overall classification accuracies obtained using 
different KNN methods based on different features 


As shown in Table 2, with individual features, classifications 
using the spectral features achieved the highest accuracies and 
those based on the textual features lowest. With all three KNN 
methods, classifications using the combined spectral, spectral, 


PA, SB, and SW stand for Norway maple, honey locust, 
Austrian pine, blue spruce, and white spruce, respectively. The 
overall accuracy (OA) is 0.82. 


This contribution has been peer-reviewed. The double-blind peer-review was conducted on the basis of the full paper. 
https://doi.org/10.5194/isprs-annals-V-3-2020-24 1-2020 | © Authors 2020. CC BY 4.0 License. 


244 


ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume V-3-2020, 2020 
XXIV ISPRS Congress (2020 edition) 


Reference 
MN LH PA SB SW Producer’s 
5 accuracy 
‘S| MN 47 1 0 0 0 0.98 
Pa LS 8 45 i 3 0 0.71 
a PA 0 0 30 0 0 1.0 
© | SB 0 4 7 25 | 5 0.61 
SW 0 4 4 8 25 0.61 
User’s 0.85 | 0.83 | 0.63 | 0.69 | 0.83 | OA: 0.77 
accuracy 


Table 5: The confusion matrix for the neutrosophic KNN and 
DS theory, where MN, LS, PA, SB, and SW stand for Norway 
maple, honey locust, Austrian pine, blue spruce, and white 
spruce, respectively. The overall accuracy (OA) is 0.77. 


5. CONCLUSIONS 


The information fusion methods developed based on the 
neutrosophic logic and DS theory for tree species classification 
was demonstrated to be promising based on the data obtained 
from WV-2 and airborne LiDAR data over the Keele campus, 
York University, Toronto Canada and the KNN classification 
method. Caution should be taken in that the classification tests 
were based on a limited number of species and training and 
testing samples, and further research with more species and field 
samples is needed. Furthermore, the KNN classification method 
was used due to its simplicity to prove concepts. Advanced 
classification methods, such as SVM, will be exploited in future 
works. 


With the proposed method, spectral, textual, and structural 
features were derived from WV-2 multispectral bands, WV-2 
panchromatic band, and LiDAR-derived CHM, respectively. For 
individual types of features, the spectral features were shown to 
be more effective in the discrimination of the five species of 
interest, namely, Norway maple, honey locust, Austrian pine, 
blue spruce, and white spruce. The textural features the least 
effective. An investigation is being carried out to derive advanced 
textural and structural features and they will be employed to 
improve the classification accuracy. The classification accuracy 
was improved by using multi-source remotely sensed data based 
on the fusion at both the feature and decision levels. For the cases 
tested, the feature level fusion performed better than the fusion at 
the decision level. For this study, the number of features from the 
combined spectral, structural, and textural remained small (27). 
The disadvantage of the fusion at the feature level might not 
evident. In future studies, more features will be used to evaluate 
the two fusion approaches. 


The results also showed that the proposed neutrosophic logic 
KNN was outperformed the commonly used weighted KNN and 
fuzzy KNN and the overall classification accuracy reached 0.82 
for the fusion at the feature level. Among the three KNN 
methods, the improvement using information fusion at the 
decision level was observed only for the weighted KNN, 
compared with the feature-level fusion. The advantage of the 
decision fusion was expected to be obvious for the cases with a 
large number of features and for combining the classifiers with 
reasonable good accuracies, in the comparison with information 
fusion at the feature level. 
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